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Abstract 


The general purpose of this study is to present post-funding Comprehensive 
School Reform (CSR) data to address the more lasting features of CSR pro- 
grams, their relation to the No Child Left Behind Act (NCLB) and support 
of its requirements, and the impact of CSR on student achievement. For the 
schools included in this study, many of the components of CSR are continuing 
to be implemented, and the majority of the schools continued to implement 
the specific CSR model, suggesting that CSR was sustained in many sites. In 
terms of student achievement, the implementation of CSR was associated with 
enhanced student achievement for this set of schools, and the student achieve- 
ment results had improved over the past four years in most schools. In general, 
the schools felt that CSR had helped the schools implement effective practices 
and programs, one of the major goals of NCLB. 

Key Words: comprehensive school reform, sustained effects, No Child Left 
Behind Act, program implementation, student achievement 


Introduction 


The United States Department of Education (2002) offers some guidance 
in the definition of Comprehensive School Reform by referring to the legisla- 
tive components of the program that, when systematically implemented, will 
lead to “scientifically based” educational practices. These components include, 
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for example, high quality professional development, measurable goals and 
benchmarks, meaningful parent involvement, external support and technical 
assistance, and proven methods for learning and instruction. Since the incep- 
tion of the Comprehensive School Reform Demonstration Program, about 
$770 million has been provided to approximately 3,000 schools across the na- 
tion implementing Comprehensive School Reform models. The program, now 
entitled the Comprehensive School Reform (CSR) Program, was reauthorized 
in the No Child Left Behind Act (Part F, Title I of P.L. 107-110). 

While CSR appears to be expanding rapidly in terms of the number of 
schools adopting programs, there are still unresolved issues relating to the ini- 
tial groups of schools that received funding. Funding began in 1998-99 and 
those schools were typically funded for a three-year cycle, which would have 
ended at the conclusion of the 2000-01 school year; thus, they would have 
completed their second year of “post-implementation” CSR funding in the 
2002-03 school year. The unanswered questions of CSR pertain to the extent 
to which those schools (1) are continuing to implement CSR models, and (2) 
have experienced sustained effects related to the models, particularly in terms 
of student outcomes. The general purpose of this study is to present post- 
funding data results to address the more lasting features of CSR programs, 
their relation to NCLB and support of its requirements, and the impact of 
CSR on student achievement. 

Background 

In one comprehensive evaluation of CSR programs, the RAND Corpo- 
ration reported mixed results (Berends, Kirby, Naftel, & McKelvey, 2001; 
Viadero, 2001). In the 163 CSR schools included in the study, about one half 
showed greater improvements in both mathematics and reading achievement 
than the district did overall. The findings in this report showed that schools var- 
ied greatly in the degree to which they had implemented CSR programs. The 
CSR program was highly implemented in only about one half of the schools 
where implementation levels were examined. Additionally, the findings seemed 
to indicate that CSR programs were more highly implemented and more effec- 
tive relative to student outcomes at smaller schools and in elementary schools. 

The notion of degree of program implementation is crucial to the evalua- 
tion of CSR programs. Some studies (see, e.g., Datnow, Borman & Stringheld, 
2000) have stressed the fact that the quality of the implementation is a signifi- 
cant factor and has a substantial impact on the success or failure of CSR reform 
models. Other researchers (Stringheld, Millsap & Yoder, 1997) have found a 
strong, positive relationship between the degree of program implementation 
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and program outcomes. Certainly, the inclusion of degree of implementation 
findings in evaluations lends credibility to the interpretation of program impact 
on outcomes, such as student achievement, and can provide further insight as 
to program-specific and school-specific differences in implementation. 

In another comprehensive evaluation of CSR, Borman, Hewes, Overman 
and Brown (2003) conducted an exhaustive meta- analysis to examine the gen- 
eral effectiveness of the CSR strategy, the effects associated with CSR, and 
differences with respect to methodological and contextual features. They found 
that there were different standards of evidence for model effectiveness in terms 
of actually improving test scores and that the number of years of model imple- 
mentation had very significant implications for understanding CSR impact 
on achievement. In particular, they noted the strong effects of CSR beginning 
with the fifth year of model implementation. Overall, the study found that the 
effects of CSR were statistically significant and meaningful relative to the stu- 
dent populations served. 

The results of previous studies of CSR in Pennsylvania also suggest that im- 
plementation of CSR in some schools does enhance student outcomes, namely 
student achievement scores (McCray, Evans, & Sheffer, 2002). A follow-up 
study of a small sample of CSR schools in the School District of Philadel- 
phia, designed to examine how changes in student achievement compare to 
schools not receiving CSR funding, indicated that although Philadelphia CSR 
schools demonstrated increases in student achievement scores during the fund- 
ing period, the rate of this increase did not differ significantly from that of 
the district as a whole (McCray, Evans, Clark, Offenberg, & Molock, 2002). 
It should be noted, however, that in terms of school context, it has been re- 
ported (Desimone, 2002) that in high-minority, low- achieving urban schools, 
systemic problems such as mobility and lack of capacity can hinder successful 
CSR implementation. 

The current study builds on the professional literature reported to date, 
particularly those efforts conducted in Pennsylvania. In examining the post- 
funding stage of program development and implementation, the study attempts 
to identify the more lasting effects of CSR. In determining the sustained effects 
that may be associated with CSR, the results of this study focus first on degree 
of implementation and then on an analysis of student achievement. 

CSR Implementation 

The data reported in this study were obtained from a total of 96 sites in 
78 schools in Pennsylvania (achievement testing occurred in more than one 
grade level at certain schools). These schools started their funding cycle at the 
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beginning of the 1998-99 school year; their funding ended at the conclusion 
of the 2000-0 1 school year. A team of consultants from the Pennsylvania De- 
partment of Education (PDE) gathered post-funding data. The intent of the 
monitoring effort was to determine the aspects of CSR that were still being im- 
plemented, the degree to which the original CSR model was still in place, and 
the approach the school was taking with respect to NCLB requirements. The 
implementation data were gathered during site visits to each of the 78 schools 
during April -June of 2003. 

The 78 schools included in this study represented 19 districts across the 
state; more than one-half of the schools, 45, were from Philadelphia School 
District (large metropolitan district serving approximately 215,000 students). 
The 45 CSR schools in the School District of Philadelphia consist of 28 ele- 
mentary schools (62%), 10 middle schools (22%), and 7 high schools (16%). 
The remaining 33 Pennsylvania schools consisted of 25 elementary schools 
(76%), 4 middle schools (12%), and 4 high schools (12%) in smaller met- 
ropolitan districts and rural areas. A total of 36 different CSR models were 
implemented: Community for Learning (CFL) was implemented in 15 schools, 
7 schools implemented CO-NECT, 6 schools implemented Talent Develop- 
ment, 5 schools implemented Core Knowledge', ATLAS and Success for All were. 
each implemented in 4 schools, while Micro Society, Balanced Literacy, Prin- 
ciples of Learning, and Lnstitute for Learning were implemented in 2 schools 
each. Also, a total of 3 schools implemented multiple models and 2 schools 
implemented locally developed models. The remaining 24 models were imple- 
mented in only one school each. 

At the time of the implementation site visit, it was reported that 50 of the 
78 schools, or about 64%, were still implementing the CSR model. Of the 45 
Philadelphia schools, a total of 26, or about 58%, were continuing to imple- 
ment the CSR model. For the other 18 districts across the state, representing 
33 schools, the CSR models were being implemented in 24 (73%) schools. 
Of the 50 schools still implementing CSR, 36 elementary schools (64%) were 
continuing to implement the CSR model; at the middle school level, 7 were 
implementing the model (50%); in the high schools, 7 continued to imple- 
ment the CSR model (63%). In comparing the models that were implemented 
in 4 or more schools, the following percentages of schools were continuing to 
implement the model: ATLAS = 75%, CFL = 73%, Core Knowledge = 72%, 
Talent Development = 67%, CO-NECT = 29%, and Success for All = 0%. 

The degree of implementation ratings were based on a four-point Likert 
scale, where 3=Implemented and 4=Functional. Across all 78 schools, the over- 
all mean was 3.87 and the standard deviation was .132. The implementation 
scores ranged from 3.52 to 4.0; a total of 30 schools were rated at the 4.0 
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level. A number of comparisons were made based on the degree of implemen- 
tation ratings. The mean score for the Philadelphia schools was 3.89 and the 
remaining schools statewide had a mean of 3.86 (an analysis utilizing the t-test 
indicated that the difference was not significant). The mean score for the ele- 
mentary schools was 3.87, for middle schools, 3.86, and for high schools, 3.89; 
the statistical test employed, analysis of variance, indicated the differences were 
not significant. Finally, the mean scores for the models implemented in four or 
more schools follow: CFL = 3.88, CO-NECT = 3.58, Talent Development = 
3.94, Core Knowledge = 3.94, ATLAS = 3.82, and Success for All = 3.94. The 
analyses of variance results indicated that the differences between the scores for 
those six models were not significant. 

Using the comments obtained in the Post-Implementation CSR Monitoring 
Assessment Instrument (developed by the Pennsylvania Department of Educa- 
tion), many of the schools in this study indicated that the CSR model helped 
prepare for NCLB implementation through its comprehensive approach to 
staff development and parental involvement, and that the things they learned 
from CSR provided a firm foundation for NCLB. Specific support for teachers 
and staff mentioned by the majority of schools included: common planning 
time, teacher release time, modeling of instructional techniques, support teams 
for shared decision making, staff development workshops, and leadership 
teams focusing on capacity building. 

In terms of employing proven strategies and methods for student learn- 
ing and teaching, most sites felt that the models they selected (CFL, Balanced 
Literacy, Talent Development, Core Knowledge, ATLAS, etc.) for implemen- 
tation were, in fact, based on scientific research. Some school district staff 
interviewed noted that the models were on the original “approved” list de- 
veloped by the U.S. Department of Education that met the criteria of being 
evidence-based and being appropriately documented in terms of proven effec- 
tiveness. As other sources of data to substantiate the scientifically based nature 
of their programs, schools cited district policies and mandates that they assume 
are backed by sufficient research. 

Another question addressed the extent to which the schools had am- 
ple evidence that their programs would significantly improve the academic 
achievement of their students. Most schools indicated that they were using 
best practices based on their CSR models, citing publications and other data 
sources related to their CSR programs. They also stated that their test results 
had improved over the past four years and these data supported the notion 
that schools were improving. They also felt that the student achievement re- 
sults were consistent with other district assessments. In general, schools felt 
that since the implementation of CSR they had become more data-driven 
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to build on strengths and improve on their weaknesses. With respect to the 
degree to which CSR provided the school with a comprehensive design, the 
schools again cited improved test scores as evidence that CSR effected many 
school-wide changes that led to improved instruction and helped invigorate 
the culture of the school. Again, schools also noted that CSR had provided 
a sound foundation for NCLB implementation and had helped the schools 
identify effective practices and programs. 


Student Achievement 

The Pennsylvania System of School Assessment (PSSA) is a statewide test 
that is administered as required under the Chapter IV Regulations of the State 
Board of Education as well as various Pennsylvania Statutes. The test is de- 
signed to assess academic levels of students, schools, and districts as well as to 
assist in identifying their strengths and weaknesses and to foster improvements 
in academic achievement. The basis of the PSSA is the content contained in 
Pennsylvania’s Academic Standards in reading, writing, speaking and listening, 
and mathematics. 

PSSA data are reported by “performance levels” and by “scaled scores.” 
Performance levels indicate the percentage of students within each criterion ref- 
erenced reading and mathematics achievement level for 5 th , 8 th , and 1 1 th grades, 
respectively. The advanced, proficient, basic, and below basic performance lev- 
el criteria were determined by Pennsylvania educators, along with parent and 
business representatives. Scaled scores are reported in reading, mathematics, 
and writing. Scaled scores are included because of their ease of interpretation 
and because they allow comparison of school results with district and statewide 
average scaled scores. Additionally, scaled scores provide one overall summary 
figure at the school level. PSSA scaled scores were used in this analysis. 

The analysis was driven by some key research questions. The first issue 
addressed was student achievement results obtained from the CSR schools 
relative to statewide results. In a comparison analysis of student achievement 
between the state of Pennsylvania as a whole and the Comprehensive School 
Reform buildings involved in this study, there exists a significant difference 
in student achievement gains as shown in Table 1 . An independent t-test on 
CSR-implementation schools and the state revealed the existence of a signifi- 
cant difference between student achievement gains in both math and reading. 
For this group of schools, the implementation of CSR seemed to be associated 
with enhanced student achievement. 
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Table 1. CSR Implementation Schools and Student Achievement: Means, 
Standard Deviations, and ^-values 



1 st Year of Funding 

Post Funding 

Student Achievement (Math) 
All CSR Schools (n= 96) 

All Pennsylvania (n= 3,057) 
t (95) = 10.56,/- < .01 

1161.04 (100.39) 
1300 (NA) 

1226.15 (102.52) 
1306 (NA) 

Student Achievement (Reading) 
All CSR Schools (n= 96) 

All Pennsylvania (n= 3,057) 
t (95) = 9.46,/- < .01 

1144.69 (106.72) 
1300 (NA) 

1204.58 (106.86) 
1303 (NA) 


Given the gains observed for the CSR schools, the next set of questions 
posed attempted to determine the more specific settings and circumstances 
where student achievement had improved. A one-way analysis of variance 
was used to determine the effects of CSR implementation and school type 
on student achievement. As shown in Table 2, the findings indicate that CSR 
implementation and school type does have a significant impact on student 
achievement in both math and reading. 

More specifically, an independent t-test on the three specific types of schools 
(elementary, middle, and high school) revealed there were no significant effects 
on student achievement in either math or reading pertaining to CSR imple- 
mentation between elementary and middle schools or between middle and 
high schools. However, the findings indicate there is a significant effect on 
student achievement and CSR implementation between elementary and high 
schools. Thus, in this sample of schools, student achievement gains were more 
likely to occur at the elementary school level than at the high school level. 

As more than 50% of the implementation sites were in the School District 
of Philadelphia, one research question addressed potential differences in the 
district setting (Philadelphia versus all other CSR schools). Using a r-test to 
determine the effects on student achievement of CSR implementation and the 
school district where implementation occurred, the results indicate that CSR 
implementation and school district does have a significant impact on student 
achievement in both math and reading. As shown in Table 3, these findings 
suggested that student achievement was enhanced to a greater degree in those 
CSR schools outside of Philadelphia. 
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Table 2. CSR Implementation, School Type, and Student Achievement: 
Means, Standard Deviations, T-values, and t- values 



1 st Year of Funding 

Post Funding 

Student Achievement (Math) 
Elementary (n= 60) 

Middle (n= 25) 

High School (n= 11) 

F (2,93) = 3.65, p < .05 

1181.83 (85.04) 
1147.20 (115.13) 
1079.09 (103.49) 

1243.33 (101.17) 
1209.60 (99.23) 
1170.00 (99.50) 

Student Achievement (Reading) 
Elementary (n= 60) 

Middle (n= 25) 

High School (n= 11) 

F (2,93) = 3.17,/? < .05 

1146.50 (99.44) 
1154.80 (122.27) 
1111.82 (111.88) 

1214.83 (96.55) 
1212.80 (107.65) 
1130.00 (135.28) 

Student Achievement (Math) 
Elementary (n= 60) 

Middle School (n=25) 
t (83)= 1.41, p>.Q5 

1181.83 (85.04) 
1147.20 (115.31) 

1243.33 (101.17) 
1209.60 (99.23) 

Student Achievement (Reading) 
Elementary (n= 60) 

Middle School (n=25) 
t (83) = .085,/- >.05 

1146.50 (99.44) 
1154.80 (122.27) 

1214.83 (96.95) 
1212.80 (107.65) 

Student Achievement (Math) 
Middle School (n=25) 

High School (n=l 1) 
f (34) = 1.102,/) > .05 

1147.20 (115.31) 
1079.09 (103.49) 

1209.60 (99.23) 
1170.00 (99.50) 

Student Achievement (Reading) 
Middle School (n=25) 

High School (n=l 1) 
f (34) = 1.97,/- > .05 

1154.80 (122.27) 
1111.82 (111.88) 

1212.80 (107.65) 
1130.00 (135.28) 

Student Achievement (Math) 
Elementary (n= 60) 

High School (n=ll) 
t (69) = 222, p <.05 

1181.83 (85.04) 
1079.09 (103.49) 

1243.33 (101.17) 
1170.00 (99.50) 

Student Achievement (Reading) 
Elementary (n= 60) 

High School (n=ll) 
t (69) = 2.50,/? <.05 

1146.50 (99.44) 
1111.82 (111.88) 

1214.83 (96.95) 
1130.00 (135.28) 
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Table 3. CSR Implementation Schools and Student Achievement in Philadel- 
phia and Other Districts: Means, Standard Deviations, and r-values 



1 st Year of Funding 

Post Funding 

Student Achievement (Math) 
Other Districts (n= 33) 
Philadelphia (n= 63) 
t (94) = 6.00, p < .01 

1231.21 (80.61) 
1124.29 (89.98) 

1300.30 (90.61) 

1187.30 (86.07) 

Student Achievement (Reading) 
Other Districts (n= 33) 
Philadelphia (n= 63) 
f (94) = 5.52, p < .01 

1222.12 (85.07) 

1104.13 (94.09) 

1277.58 (90.17) 
1166.35 (94.75) 


In terms of specific CSR models and their impact on student achievement, 
a one-way analysis of variance was used to determine the effects of CSR im- 
plementation. The results, as shown in Table 4, indicate that Comprehensive 
School Reform, as a whole, has a significant effect on student achievement, 
but a few specific CSR models had a greater impact on student achievement 
in both math and reading than others. Keeping in mind the relatively small 
sample size with respect to the number of schools in which some models were 
implemented, the particular CSR models that demonstrated a significant effect 
on student achievement were Community for Learning (in reading and math) 
and Core Knowledge (in reading) . 

The final research question addressed the potential relationship between the 
degree of model implementation and student achievement. Schools were rated 
on the degree of implementation and comparisons made between schools that 
were rated as “fully implemented” (with scores of 4.00 on the four-point scale) 
versus the schools rated as “implementing” (with scores < 4.00 on the four 
point scale). While the results consistently favored the “fully implemented” 
schools, there were no statistically significant differences on student achieve- 
ment in math and reading (see Table 5). 


23 


THE SCHOOL COMMUNITY JOURNAL 


Table 4. CSR Implementation, CSR Model Type, and Student Achievement: 
Means, Standard Deviations, F-values, and f-values 



1 st Year of Funding 

Post Funding 

Student Achievement (Math) 
CSR Model in 3+ Sites (n=60) 
All other CSR Models (n=36) 
F (75.69) = 207, p > .05 

1164.33 (109.95) 
1155.56 (83.24) 

1227.83 (104.33) 
1223.33 (101.00) 

Student Achievement (Reading) 
3 > CSR Model Sites (n=60) 
All other CSR Models (n=36) 
A (94)= 2.143, />>.05 

1149.33 (112.55) 
1136.94 (97.27) 

1206.83 (112.00) 

1200.83 (99.20) 

Student Achievement (Math) 
CFL (n= 15) 

Other CSR Models (n= 81) 
f (94) = 624, p < .01 

1280.67 (69.02) 
1138.89 (89.20) 

1334.67 (55.66) 
1206.05 (96.55) 

Student Achievement (Reading) 
CFL (n= 15) 

Other CSR Models (n= 81) 
f (94) = 5.492,/- < .01 

1279.33 (49.92) 
1119.75 (95.18) 

1329.33 (53.25) 
1181.48 (98.01) 

Student Achievement (Math) 
Core Knowledge (n= 8) 

All other CSR Models (n= 88) 
t (94) = 1.940,/- > .05 

1230.00 (92.27) 
1154.77 (99.22) 

1292.50 (85.48) 
1220.11 (102.21) 

Student Achievement (Reading) 
Core Knowledge (n= 8) 

All other CSR Models (n= 88) 
t (94) =2.123,/- < .05 

1215.00 (95.32) 
1138.30 (105.88) 

1280.00 (84.52) 
1197.73 (106.41) 


Table 5. Student Achievement, Full CSR Implementation Sites, and Func- 


tional Implementation Sites: Means, Standard Deviations, and f-values 



1 st Year of Funding 

Post Funding 

Student Achievement (Math) 
Full Impl. CSR Sites (n= 40) 
Implemented CSR Sites (n= 56) 
t (82.24) = 1.880,/- > .05 

1186.75 (106.52) 
1142.68 (92.41) 

1249.25 (103.34) 
1209.64 (99.58) 

Student Achievement (Reading) 
Full Impl. CSR Sites (n=40) 
Implemented CSR Sites (n= 56) 
f (94) = 1.455,/- > .05 

1168.75 (113.59) 
1127.50 (99.00) 

1223.25 (116.91) 

1191.25 (97.96) 
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Discussion 

Clearly, the majority of schools continued to implement the CSR model, 
and the evidence suggests that CSR was sustained in many sites. The degree 
of implementation ratings provided more compelling evidence related to the 
more lasting features of CSR programs. All schools were highly rated on the 
post-implementation instrument, with the scores falling between 3.5 and 4.0 
on the four-point scale. In terms of implementation results, the initial imple- 
mentation data gathered during the funding cycle suggest that most CSR 
features were implemented in most schools. On the same four-point scale, 
where 3=Implemented and 4=Functional, most schools had mean scores across 
all CSR components that were between 3 and 4. The distribution of post- 
implementation scores was very similar to the initial ratings, and, in general, 
the ratings were slightly higher for the post-implementation assessment. 

For those sites that did not continue implementing the CSR model, there 
were a number of reasons given by schools for their decision to discontinue. 
The most frequent cause, most notable in Philadelphia, related to district man- 
dates regarding the specific scope and sequence to be followed that were seen 
as incompatible with the existing model in the school. Another general issue 
related to funding, where schools indicated that the programs (Talent Devel- 
opment, Montessori, Success for All, ATLAS, and Micro Society) were very 
expensive and that they did not have adequate funds to continue supporting 
the CSR model. A number of schools also mentioned a shift in focus or empha- 
sis in the district to balanced literacy approaches and a movement away from 
specific CSR models. There were also two reasons that were model-related: a 
teachers’ union grievance forced three sites to discontinue the CFL model, and 
concerns about the lack of services from the provider was noted by three sites 
that discontinued the CO-NECT model. 

While the trends across all schools reflect positively on CSR programs and 
their sustainability, the data does not provide persuasive evidence in terms of 
guaranteeing that any given model will continue to be highly implemented 
after the funding cycle has been completed. A much larger sample of schools 
implementing CSR models and longer-term data would make more conclusive 
judgments possible regarding the sustainability of reform models, and particu- 
larly their effects on school performance and student achievement. While the 
degree of CSR implementation data for the 78 schools included in this study 
indicated that the schools, for the most part, were in compliance with state and 
federal guidelines, more in-depth, systematic implementation data would be 
helpful in future studies. 

In terms of student achievement, the implementation of CSR was associat- 
ed with enhanced student achievement for this group of schools. These results 
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are similar to those presented by Borman et al. (2003), who reported that the 
effects of CSR on student outcomes were statistically significant and influential 
relative to the student populations served. Also, in this sample of schools, stu- 
dent achievement gains were more likely to occur at the elementary school level 
than in high schools. These results support the findings presented by Berends 
et al. (2001), who found that CSR programs were more effective relative to 
student outcomes at smaller schools and in elementary schools. However, with 
the small number of high schools included in this study, it may be that future 
studies should look more specifically at CSR implementation in high schools. 

These findings suggested that student achievement was enhanced to a great- 
er degree in those CSR schools outside of Philadelphia. McCray, Evans, Clark, 
et al. (2002) reported in a previous study of Philadelphia CSR schools that 
while student achievement scores increased, the gains did not differ significant- 
ly from that of the district as a whole. It may be that future studies should more 
closely examine urban schools receiving CSR funding and focus on school con- 
text variables that can pose barriers to successful CSR implementation. 

In this sample of schools, the specific CSR models that demonstrated a 
significant effect on student achievement were Community For Learning and 
Core Knowledge. Because of the small number of schools included in the 
current study, it is difficult to conclude with any degree of confidence that a 
particular model will assure success. In the CSR study conducted by Borman et 
al. (2003), they found that there were different standards of evidence for model 
effectiveness in terms of actually improving test scores, and additional studies 
in this regard may be worthwhile for the CSR schools. Certainly, studies with 
larger numbers of implementing schools by model would enable researchers to 
better gauge the impact of particular CSR models. 

The results related to degree of CSR implementation and student achieve- 
ment suggest that the “fully implemented” schools tended to do better than the 
remaining set of schools, although the results were not statistically significant. 
Thus, while it was not possible to offer strong and convincing evidence that 
would confirm the results presented by Datnow et al. (2000) and Stringfield et 
al. (1997), who found that the quality of the implementation has a consider- 
able impact on the success or failure of CSR reform models and their impact 
on program outcomes, the data do support the general notion. 

It may be that future evaluations of implementation should consider the 
use of multiple indicators or measures that more comprehensively address all 
of the components of CSR included in the most recent legislation and enable 
investigators to examine program-specific and school-specific differences in 
implementation. In that sense, future studies might expand to include imple- 
mentation data from schools as they complete the funding cycle and examine 
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the relationship between the degree of implementation of specific CSR models 
and student achievement. Future evaluations might also consider additional 
measures related to student outcomes and utilize both qualitative and quan- 
titative methods to assess the success of school reform efforts. Studies such as 
those would be useful in providing evidence documenting the degree to which 
the CSR model is being successfully implemented and to provide confirmation 
that, as a result of the successful implementation of the CSR model, anticipat- 
ed improvements in student achievement are being attained and sustained. 

Conclusion 

The purpose of this report was to present post-funding data results to ad- 
dress the more lasting features of CSR programs and their relation to NCLB 
and support of its requirements. In general, many of the components of CSR 
are continuing to be implemented in the schools and are supporting NCLB 
requirements. The majority of the schools included in this study continued to 
implement the specific CSR model. The degree of implementation ratings pro- 
vided further evidence related to the more lasting features of CSR programs, as 
all schools were highly rated on the post-funding implementation instrument. 

The findings from this study indicated that the CSR model helped schools 
prepare for NCLB implementation through its comprehensive approach to 
staff development and parental involvement, and thus the experience with 
CSR provided a firm foundation for NCLB. In terms of employing proven 
strategies and methods for student learning and teaching, most schools felt that 
the models they selected for implementation were, in fact, developed through 
scientifically based research. 

In terms of student achievement, the implementation of CSR was associ- 
ated with enhanced student achievement for this set of schools. Also, in this 
sample of schools, student achievement gains were more likely to occur at the 
elementary school level than in high schools. In addition, these findings sug- 
gested that student achievement was enhanced to a greater degree in those 
CSR schools outside of the large, urban District of Philadelphia. Finally, there 
seemed to be a positive trend between student achievement outcomes and the 
implementation scale ratings of the schools. 

Most schools indicated that they were using best practices based on their 
CSR models and that test scores had improved over the past four years. They 
also felt that the student achievement results were consistent with other district 
assessments and that schools had become more data-driven since the imple- 
mentation of CSR. In general, the schools felt that CSR had helped the schools 
implement effective practices and programs, one of the major goals of NCLB. 
In conclusion, the major components of CSR programs were sustained, and 
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experience with CSR programs helped support the implementation of NCLB 
requirements in the participating schools and districts across the Common- 
wealth of Pennsylvania; however, with the large amount of funding allotted 
by the United States Department of Education for schools using CSR models, 
more national analyses of CSR models are needed to assess their relationship to 
student achievement in terms of standardized tests and practical use. 
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